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ABSTRACT 

A  class  of  finite-memory  deterministic  algorithms  is  introduced  and 
investigated,   Optimiim  algorithms  are  found  for  a  small  number  of  states 
(up  to  21)  and  an  asymptotic  bound  on  error  probability  is  obtained  for 
a  large  number  of  states.   The  algorithms  provide  their  own  stopping 
rule. 
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I.   INTRODUCTION 

Computers  seem  to  grow  more  complex  and  more  sophisticated  on  almost 
a  continuous  basis.   A  logical  future  step  in  computer  development  V70uld 
be  to  provide  the  computer  with  some  decision  making  capability.   This 
capability  of  necessity  would  be  limited  by  the  size  of  the  computer  core. 
Such  a  property  would  be  of  particular  value  if  the  computer  were  design- 
ed to  operate  without  human  assistance.   For  example,  exploration  of  the 
nearer  stars  could  most  easily  be  accomplished  by  unmanned  spacecraft. 
Yet  the  immense  distances  preclude  human  involvement  in  any  decision  pro- 
cess.  If  a  computer  with  a  decision  making  capability  were  to  be  used 
within  the  spacecraft,  it  would  almost  certainly  be  small  and  of  very 
limited  core  size.   Such  an  automaton  would  be  required  to  make  decisions 
with  minimum  probability  of  error  and  constrained  by  available  memory. 
One  form  of  a  decision  process  could  probably  be  adapted  from  the  statis- 
tical test  of  hypothesis. 

In  testing  a  hypothesis,  the  statistician  normally  forms  the  likeli- 
hood ratio  and  reduces  the  ratio  to  a  sufficient  statistic  which  is  to  be 
less  than  or  greater  than  some  constant  k.   For  a  sample  of  size  n;  a 
(Probability  of  Type  I  error)  and  p   (Probability  of  Type  II  error)  will 
exponentially  approach  zero  as  n  becomes  large.   To  apply  the  procedure 
at  time  n  means  that  sufficient  memory  must  be  available  at  time  n  to 
record  the  observations  }^  ,}^  ,]^  , . . .  ,X  .   In  even  the  simplest  cases  this 
memory  must  grow  indefinitely  with  time.   Summarizing  the  data  in  a  suf- 
ficient statistic  does  not  insure  that  the  memory  requirement  will  be  re- 
duced.  The  sufficient  statistic  is  data  reducing  only  in  that  it  maps 
the  observations  from  R  to  R,  but  the  cardinality  of  the  memory  may  be 


at  least  as  great.  For  example,  suppose  that  an  experimenter  wishes  to 
estimate  |i  for  a  Normal  random  variable  and  uses  x  as  a  sufficient  sta- 
tistic.  Further  suppose  that  in  99  repeated  trials  the  experimenter  finds 

that  Sx  =  1.000.   Then  x  =  1.0/99  =  0.0101 ,  and  the  memory  requirement 

has  become  infinite, ,  It  would  be  tempting  to  round  off  the  statistic  to 
some  finite  dimension;  however,  Cover  [l]  has  shown  that  a  and  p  then 
do  not  tend  to  zero.   Additionally,  the  rounded  off  statistic  may  not 
converge  to  the  same  distribution  as  the  estimated  parameter.   If  con- 
strained by  finite  memory,  some  other  model  must  be  devised.   One  possible 
approach  is  to  use  only  the  last  k  observations.   This  idea  has  been  in- 
vestigated by  Robbins  [2], 

Although  not  originally  intended  as  a  method  for  testing  hypotheses, 
Robbins'  model  maximizes  the  long  run  expected  number  of  "successes"  given 
two  alternative  courses  of  action  and  finite  memory.   Suppose  that  an 
experimenter  has  two  coins  and  that  he  wishes  to  maximize  the  number  of 
heads  thrown  during  a  sequence  of  tosses.   The  minimum  variance  unbiased 
estimator  of  p  is  x  and  as  the  number  of  repeated  trials  increases,  the 
variance  of  the  estimator  goes  to  zero.   Therefore  if  an  experimenter 
had  prior  knowledge  of  the  probability  of  heads  for  each  coin  he  would 
use  the  coin  with  the  greater  probability  of  heads  exclusively  and  know 
with  certainty  that 

Limit   number  of  heads  in  first  n  tosses  =  max(pi  ,P2) 
n-^  n 

where  p.  =  probability  of  heads  for  the  i    coin.   Without  prior  know- 
ledge, and  constrained  by  a  finite  memory,  the  experimenter  must  decide 

which  coin  to  use  on  the  basis  of  the  results  of  the  previous  r  trials. 
Robbins  formulates  a  decision  rule  in  the  proof  of  the  following  theorem: 
"Define  the  rule  R  as  follows:   start  tossing  with  coin  1.  Stop 


if  the  first  toss  is  tail,  otherwise  continue  tossing  until  the  first 
run  of  r  successive  tails  occurs  and  then  stop.   This  defines  the 
first  block  of  tosses  with  coin  1.   Now  start  tossing  with  coin  2 
and  apply  the  same  rule,  obtaining  the  first  block  of  tosses  with 
coin  2.   Then  st^art  again  with  coin  1  and  apply  the  same  rule,  obtain- 
ing the  second  block  of  tosses  with  coin  1,  and  so  on  indefinitely, 
thus  generating  an  infinite  sequence  of  tosses  consisting  of  alter- 
nate blocks  of  tosses  with  coins  1  and  2.   With  rule  R  so  defined, 

r 

we  assert  that 

Limit   number  of  heads  in  first  n  tosses    _ 
n-»oo  n 

r       r 
Pi  qp  +  Pp  qi   ,. 
r    r 

qi  +  qs 

Note  that 

r       r 
Limit   Pi  qs  +  B3  qi   =  max  (pi  ,I^  ) 


];— »oo 


r    r 

qi  +  qs 


Using  methods  similar  to  the  Robbins  model.  Cover  [ l]  developed  a 
4-state  memory  algorithm  for  testing  the  hypothesis  p  >  Po  vs  p  *^  Po  , 
given  a  sequence  of  iid  Bernoulli  random  variables.   In  the  Cover  model, 
the  pair  (T,Q)  can  take  values  in  {0,l}.   T  keeps  track  of  the  currently 
favored  hypothesis  and  Q  records  the  results  of  the  current  run  test. 
TVo  sequences  [s.}i  and  [r.}i  of  positive  intergers  are  considered.   The 


sequences  of  observations  are  divided  into  blocks  S, ,  R, ,  S^,  R„ , . . . . 

where  S,  denotes  the  first  s.  observations,  the  next  r.  by  R,  ,  and  so  on. 
1  1  '  1   -^   1' 

T  is  initially  arbitrary;  if  all  observations  in  a  block  S.  are  equal  to 
1,  Q  is  set  to  1.   If  all  observations  in  a  block  R.  are  equal  to  0  ,  Q 
is  set  to  1 .   At  the  end  of  each  block  the  currently  favored  hypothesis 


is  updated  by  the  rule 

T  =1   if  Q  =  1  and  n  is  at  the  end  of  an  S  block 
n 

=0   if  Q  =  1  and  n  is  at  the  end  of  an  R  block 

=  T  ,   otherwise 
n-1 

The  lengths  of  the  blocks  of  S's  and  R's  are  determined  as  a  function  of 
p  ,  and  Cover  shows  that  the  limiting  probability  of  error  under  either 
hypothesis  is  zero,   (See  Figure  1). 


Figure  1. 
A  two  state  Markov  chain  where  T  can  take  on  values  in  {0,l}. 

Although  the  memory  size  in  Cover's  model  is  now  finite  the  updating  rule 
still  depends  on  n. 

The  first  genuine  finite  memory  model  has  been  proposed  by  Hellman  and 
Cover  [3], [4].   They  proposed  a  family  of  algorithms  of  the  type 

T^  =  f(T^_3^,x^)    ;   f:   {l,2, .  .  .  ,m}  XX -[  1,2, .  .  .  ,m} 
d^  =  d(T^)        ;   d:   {l,2,...,m}  -  {K,?!} 

T  denotes  the  statistic  at  time  n,  x  is  the  value  of  the  n   sample, 
n  '   n  ^ 

f  is  a  transition  function  and  d  is  a  decision  function.   Note  that  T 
is  of  finite  memory  since  Te  {l,2,...,m};  and  given  an  initial  value  of 
the  statistic,  the  sequence  T   forms  a  Markov  chain  over  the  state  space 
M  =  il,2,...,m}.   The  goal  is  to  minimize  the  expected  asymptotic  pro- 
portion of  errors 


P(e)  =  E  {Limit  ^  Ye.  } 


n-~o   "i^ 
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where 

.1   if  d,  ^  H, 

e.  =  I 

^     0  if  d.  =  H 

1    true 


i    true 


Hellman  and  Cover  have  established  a  lower  bound  for  the  proportion  of 
of  error.   Let  Tin   and  Tf^  denote  the  prior  probabilities  of  the  null  and 
alternate  hypotheses.   Let  £^  and  f_  be  the  probability  densities  of  the 
sample  under  the  respective  hypothesis  with  respect  to  a  dominating 
measure.   Define  the  likelihood  ratio  to  be  j^  (x)  =  L^^x) /  f^- (x)  .      Let 
i   denote  the  ess  sup  of  the  likelihood  ratio  and  Z_   the  ess  inf  where  the 
supremum  and  infimum  are  taken  over  all  measureable  sets  with  positive 
dominating  measures.   Define  Y  -  ^f^-      Then  for  an  irreducible  m-state 
automaton,  P(e)  ^  P*  where 


P*  = 

y^-^    -  1 


if  y"'^^  max  {V%'%/^ 


-4V%} 


=  mimTL^jTU  ?■  otherwise. 

Hellman  and  Cover  further  prove  that  a  reducible**  (m+1) -state  automaton 
obeys  the  same  bound  on  P(e)  as  an  irreducible  m-state  automaton. 

If  the  prior  probabilities  of  the  null  and  alternate  hypotheses  are 
equally  likely,  that  is  if  ^-ir  -  ^^  -   %>  then  for  an  irreducible  m-state 


automaton 


y"^-^  -  1 

y^(r"-I)  _  1 

(m-1)   , 
Y     -  1 


Y^n.-!)  ^  ^ 


**  We  call  the  automaton  reducible  (irreducible)  if  the  Markov  chain  [t  ] 
is  reducible  (irreducible). 
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If  the  m-state  automaton  is  reducible,  the  bound  becomes  at  least  as 
great  as 


P(e) 


%(m-2) 
Y       +  1 


case 


In  the  case  of  the  Bernoulli  trials,  consider  the  two  hypotheses 

5C  :  p  =  pj^ 

IT  :  p  =  pu.     where  ^v  =  ^rr  =  ^ 

Without  loss  of  generality  it  may  be  assumed  that  IV  >  p.   in  which  case 

-  -     P^G" 

i   =   pj^p^  ;  i,  =  qj^q J  ;  and  Y  =  ^/^  =  "  • 

Further,  if  the  hypothesis  if  symmetric,  that  is,  if  pu^  =  1-p^j  then 

Y  =  { — J  ;   for  an  irreducible  m-state  automaton 

P(e)  ^  -^ 


^n; 


m-1 


and  for  a  reducible  automaton 

1 


P(e)  ^ 


^n- 


m-2 


While  the  lower  bound  cannot  be  achieved  except  in  degenerate  cases, 
Hellman  and  Cover  demonstrate  an  e -optimal  class  of  automata,  that  is, 
for  every  e>0  there  exists  an  automaton  such  that  P(e)  ^  P*  +  e . 
Define: 

K^=  [x  eX  :i(x)  ^  [(1/7)  +  e]"^ 

?g=  [x  eX  aix)   ^   a  +   e)] 

S,     =  fx  eX  :   x  ^  (K  U3:  )] 

Let  the  transition  function  f,  be  specified  as  follows  (see  figure  2): 
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f(i,x) 


i  +  1 

i 

i  -  1 


fm-l 


f(l 
f(m,x)  =■ 


X  e  5^ 
X  e  4 
X  e  ? 


for  2  ^  i  ^  m-1 


with  probability  6  >  0  if  x  e  5C 
otherwise 

with  probability  k6  >  0  if  x  e 
otherwise 


I    ^£   2   ^c   3 


:ft=A±3» 


^  m-l  ^jr^  m 


Figure  2. 

Transitions  are  made  to  adiacent  states  only  when  the  events  K  or  JJ 

-"  ■'  e     e 

are  observed.   Thus  the  automaton  enters  an  end  state  only  on  strong 
evidence  to  support  that  hypothesis.   If  6  is  allowed  to  become  arbitrar- 
ily small,  then  the  automaton  tends  to  leave  the  end  state  with  a  very 
low  probability.   Decisions  made  in  the  end  states  have  the  least  proba- 
bility of  error,  so  as  6  -•  0,  the  P(e)  should  asymptotically  approach  P*. 

While  the  Hellman-Cover  algorithm  is  useful  in  producing  sequences 
of  decisions,  the  algorithm  is  not  easily  adapted  to  situations  in  which 
only  a  single  decision  is  required.   The  irreducible  automaton  will 
asymptotically  approach  the  lower  bound  for  probability  of  error  after 
a'large  enough' number  of  observations;  however,  there  is  no  easily  de- 
fined rule  which  would  specify  when  this  number  had  been  reached.   It 
should  also  be  noted  that  the  Hellman-Cover  automaton  requires  artificial 
randomization  for  transitions  out  of  the  end  states.   Some  ancillary 
mechanism  must  be  provided  to  achieve  this  desired  randomization.   In  the 
case  of  a  small  computer,  additional  core  storage  would  probably  be 
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required.   The  closer  P(e)  is  to  approach  P*,  the  smaller  is  the  prob- 
ability 6  which  must  be  generated  -  which  requires  even  more  additional 
core  storage.   It  is  therefore  believed  that  there  are  strong  pragmatic 
reasons  for  adopting  an  algorithm  with  absorbing  states  despite  higher 
asjTmptotic  probability  of  error. 

In  this  paper  a  special  class,  a  ,  of  symmetric  (2n  +  3)-state  al- 
gorithms with  two  absorbing  states  will  be  developed.   Derivations  and 
proofs  within  the  paper  are  restricted  to  symmetric  Bernoulli  random 
variables,  but  it  would  also  be  feasible  to  extend  these  concepts  to 
non-symmetric  hypotheses  and  to  distributions  other  than  Bernoulli. 
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II.   DESCRIPTION  OF  THE  ALGORITHM 

Let  X,,  X^ ,  X-,,...  denote  a  sequence  of  independent  identically  dis- 
tributed Bernoulli  random  variables  which  can  take  on  values  H  or  T. 
Consider  two  hypotheses,  K   and  ^   with  equal  prior  probabilities  and  such 
that 

P(Xj^  =  h|K)  =  P(X^  =  Tp)  =  p,  where  %  <  p  <  1. 

As  the  notation  suggests,  the  sequence  of  random  variables  can  be  thought 
of  as  successive  tosses  of  a  coin  which  is  biased  towards  Heads  under 
hypothesis  JC  or  biased  towards  Tails  under  hypothesis  3"  . 

Define  the  algorithm  (M,f,d)  (See  Figure  3)  such  that  M  =  [-(n+1),  -n, 
...,  -1,0,1, .. .n, n+1 j  with  ±(n+l)  the  two  absorbing  states  and  0  the  init- 
ial state;  d(n+l)  =  JC,  d(-(n+l))  =51  ,  otherwise  arbitrary;  and  the  tran- 
sition function, f,  such  that 

f(s,H)  =  s+1,    f(s,T)  =  s-p(s)     if  s  =  1,2,. ..,n 
f(s,T)  =  s-1,    f(s,H)  =  s-fp  (s)     if  s  =  -l,...,-n 
f(s,H)  =  1,     f(s,T)  =  -1        if  s  =  0 
f(s,H)  =  s,      f(s,T)  =  s         if  s  =±(n+l) 
The  integers  p (1) , . . . ,p (n)  satisfy  the  inequality  1  ^  p (s)  ^  s. 


d=^     H       H     " 


Start  .  -1/ 

^    d  =  A 


.(n+i)  -n     -s    -s+j)U)   -I    0    I     s-j)(s)   s      n    n  +  1 


An  Algorithm  f  =  (p(l), ,p  (n))  e  a 

Figure  3. 
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The  specific  form  of  the  algorithm  (M,f,d)  will  henceforth  be  denoted 
f  =  (p  (1)  >  •  •  •  >p  (fi))  •  Figure  4  shows  the  algorithm  and  the  transition 
matrix  for  the  case  when  n  =  4  and  f  =  (1,1,2,2). 

5   4   3   2   1   0-1-2-3-4-5 
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Figure  4:   The  transition  matrix  for  the  case  where  n  =  4  and  f  =  (1,1,2,2) 
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III.   THE  e -OPTIMAL  STOCHASTIC  AUTOMATON 

Although  the  class  a   consists  of  deterministic  algorithms  only,  it 
can  be  easily  shown  that  with  randomization,  the  Hellman-Cover  lower 
bound  could  be  approached  arbitrarily  closely.   With  the  algorithm 
(M,f,d),  the  probability  of  error  can  be  written 

?     =  k   Pr  (absorption  at  -(n+l)|K)  +  ^  Pr  (absorption  at  n+l|ir). 

Let  P. (K)  denote  the  absorption  in  state  i  without  return  to  j  given  that 
hypothesis  K   is  true  and  P.(«^)  denote  the  corresponding  probability  given 
tS,   where  i  =  ±(n+l).   Then  since  0  is  a  recurrent  event 

^e  "  ^  ^-(n+1)^  "^  ^  ^n+l^"^   "^^^^  ^^  symmetry 
Given  K,  we  know  that  P°   .i\CH^)  +  P°  i^O  =  1,  so 


'    ^-(nrt)W+P:+lW 


p°  ,   00  -,  -1 


Let  f  =  (l ,2, 3, . . . ,n)  and  1  >  5  >  0  and  consider  the  absorption  at  n+1 . 
If  a  Head  is  observed,  move  to  the  next  higher  state  with  probability  6, 
otherwise  remain  in  that  state.   If  a  Tail  if  observed,  return  to  0  since 
p (s)  =  s  for  all  s.   Since  return  to  0  is  a  recurrent  event, 

o   _^     n+1.  n+1   fr\+l\    n+2^n+l.^  ..  ,  f  n+2\    n+3.n+l..  c^2 
n+1^  "P  \1/P       ^  ^^   \  2  yP   ^    ^   ^^^  " 

n+1. n+1  r     fn+l\     .^    ..  ,  /n+2\  2..  ..2  , 

=  p     6      L       \  1  yp(^"^^  '^  \  1  F  ^  ^^       '■• 
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k=0 


n+l-n+1   ri  t^    an  i"" 

=  P       6         [1   -   P(1-6)J 


=  P        o  ,        (q  +  Op) 


Similarly, 

-(n+1)^    =   q        5  (p  +  6q) 

Thus 

e         L  n+1,      ,    ^    .n     J 

q      (q  +  op) 


Taking   the    limit   as   6  -»  0,   we  have 

^2n+l 

•  q 


.2n+l        -1 


which  is  the  Hellraan-Cover  lower  bound.   Unfortunately  this  e -optimal 
automaton  is  of  little  practical  use  since  the  expected  time  to  absorp- 
tion becomes  infinite. 
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IV.   DETERMINATION  OF  ERROR  PROBABILITY 

To  obtain  an  explicit  expression  for  the  probability  of  error  in 
terms  of  the  algorithm,  we  will  first  prove  the  following  proposition. 

1 

Proposition  1:   Let  f  =  (p (1) , . . . ,p (n)) 

n+1     -.  -1  F  (^'f) 

.q/     n^  'J      n      F  (p,f) 


tten    P,4w(^)   R„(f)]   .\(0'^^^  (2) 


Where  F(x,f)  is  a  polynomial  in  x  of  degree  less  than  or  equal  to  n  and 

with  integral  coefficients.   With  initial  conditions  F  ,  =  0  and  F  =  1, 
°  -1  o 

these  polynomials  satisfy  the  recurrence  relationship 

F„(-.«  =  ^^.i^-.O    -    (l-x)/(">F_^.^.p(„,(x,f)  (3) 

where  n  =  1,2, . . . . 

Proof: 

From  (1)  we  know  that 

p  =ri  +  ^""^      ^ 


^-(n+1)^ 


Next  note  that 

■  ^n+1  ^  "  P^n+1^   ^"""^  ^^""^ 

P  ,1     00  =  PVi    where  v.,    is'  the  expected  number  of  visits  to 
n+1    ^^l,n         l,n 

n  before  a  visit  to  n+1  or  0  given  that  the  chain  was  started  in  state  1. 

From  basic  properties  of  Markov  chains  we  know  that  V,    [5]  is  the 

l,n 

f"h  1 

(l,n)   entry  of  the  fundamental  matrix  M  =  (I-Q)   where  Q  =  [p. .1  is 
an  n  X  n  matrix  with  entries  p . .  =  p  if  j  =  i+1,  p..  =  q  if  j  =  i-p(i); 
and  p. .  =  0  otherwise.   (See  Figure  5). 
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Figure  5:   The  matrix  Q  for  the  case  when  n  =  6  and  f  =  (1,1,1,2,3,3) 
Applying  the  formula  for  the  inverse  of  a  matrix,  we  have 


wh 


ere  ]  *  |  is  the  determinant  operation  and  |  (I-Q)  .   ,.|  is  the  determin- 


th 


ant  of  the  (l,n)  cofactor  transposed.  By  deleting  the  first  column  and 
n  row  of  the  matrix  (I-Q),  the  submatrix  (I-Q)  .  ,v  is  lower  triangular 
with  entries  -p  on  the  diagonal o   Hence  its  determinant  is  equal  to  (-p) 


Substituting. 


V,  „  =  (-P)""^-1)"-'' 

1 ,  n 


n-1 


I-Ql 


-1 


=   P    ll-Ql"' 

If  we  denote  |l-Q|  =  F  (p,f)  and  repeat  the  entire  argument  with  p  and  q 
interchanged,  then 

Multiplying  and  substituting  into  (1),  we  have  that 

r     /pN'^-*-^  ^n^^'f)  r^ 

P  =   1  +  (■'-)    — — z — 7T       which  was  to  be  proved. 
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To  establish  the  recurrance  formula  for  F   (see  Figure  6),  expand  the  deter- 
tninant  along  the  n   row.   If  p  (n)  =  n,  then  the  n   row  has  all  zeroes 

except  a  1   on  the  diagonal;  F  =  F   ,  and  (3)  holds.   If  p (n)  <  n,  expan- 

th 
sion  along  the  n   column  gives 

F  (p,f)  =  F   ,'(p,f)  +  p|i-q|  ,   1   V 
n  '^^      n-1         '   ^' (n-l,n) 

Expand  |I-Q|  ,     .  along  the  last  column  and  repeating  this  p (n)-l  times 
^n-  i ,  n^ 


yields 


F  (p,f)  =  F    (p,f)  +  pP^"^  D 
n         n-  i 


The  determinant  D  has  all  zeroes  in  the  last  row  except  the  diagonal  entry 
which  is  -q.   Therefore, 

D  =  -qF   ,   ,  . (p, f)   and 
^  n-l-p (n) ^^'  ' 

F  (p,f)  =  F   ,(p,f)  -  qp^^'^^F   ,   .  s(p,f) 
Checking  the  initial  conditions  completes  the  proof. 


F5(P,f) 


F5(p,f) 


1  -p  0  0  0 

-q  1  -p  0   0 

-q  0  1  -p   0 

0  -q  0  1  -p 

0  -q  0  0   1 


F^(p,f)  +  p 


1  -p  0  0 

-q   1  -p  0 

-q   0  1  -p 

0  -q  0  0 


1  -p   0 


F  (p,f)  +  p   -q   1  -p 
I  0  -q   0 


=  F^(p,f)  +  p  qF^(p,f) 


Figure  6.   The  recurrance  relationship,  f=  (1,1,2,2,3)  . 
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Next,  call 


R  *  =  max  R  (f)  =  R  (f*) 
n    ^    n       n 
tea 
n 


Then  from  (2) 

n+1    -,-1 


'  P     ,   .n+1    -,- 
P  *  =  min  P  =|l+("^)   R*J 


If  we  take  f  =   l,2,3,...,n  ,  then  R  (f)  =  1  so  that  R  *  2:  l 

'  n  n 


The  Hellman-Cover   lower  bound   states    that 

P  ,    .n+l  -,-1  r-  ,    v2n+l  -,-1 


/    vn+1  .    s2n+l  /   xn 

which   implies    that  l-^j        R  *  ^  (-^j  and   R  *  ^  f-^j 


n 

Thus    1  ^  R 

\q/ 


n  \q/ 
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V.   REFINING  THE  BOUNDS  FOR  R 
n 


In  the  preceeding  paragraph,  it  was  shown  that  1  ^  R  *  ^  i^j    .      It 


would  now  be  useful  to  see  if  R  *  exists  in  a  form  which  can  be  expressed 

^  n 

as  a  limit  for  large  n.   To  do  so,  proposition  2  is  proved. 

Proposition  2: 

Let  f.  =  <p  .  (1)  , , . ,  ,p  ,  (n.  )>€a    for  i  =  1,2  ,  and 

f  =  (p  (1)  , . . .  ,p  (n^  +  n_))ea   ,    be  such  that  p^(s)  <  s  for  s  >  1. 
i    Z    n-i+no  z 

Then  R  *,    ^  R   (f,)R   (f„). 
n,+n     n^^  1   n^   2 

Proof: 

p  \{k)  if  k  =  1, .  . .  ,n-j^ 

Let  p  (k)  =  n]^  +  1  if  k  =  nj^  +  1 

P2(k-nj^)  if  k  =  n-  +  2,...,n,  +  n_ 

First  prove  by  induction  on  n„  that 

F      (x,f)  =  F   (x,f  )F   (x,f  )  (4) 

n^+n  n,     in-    2. 

If  n_  =  1,  then  F   =1.   But  since  p  (n-.+l)  =  n^+1,  we  know  that  (4)  holds, 
If  (4)  is  true  for  all  n  ^  n,  then  if  n„  =  n+1  from  the  recurrance  rela- 
tionship we  know  that 

Po(n^) 

F  ^   (x,f)  =  F  ^  (x,f)  -  X      (l-x)F  ^  ^     .      .(x,f) 
n^^+n^V'      n  +n^  '  ^  ^        '    n^^+n-p^Cn^) 

By  the  induction  hypothesis, 

V-^"*'^  =  \<^'^l>^n<^'f2>    -^^ 


F  ^     r.    t      N(x,f)  =  F   (x,f,)F     ,     N(x,f^) 
^1  ^'P2^"2^         "l       n-p2(n2)^   V 


If  ^2^^'}}    ^  '^^    ^"^^^ 


F   ,   (x,f)  =  F     (x,f)  -  X      (l-x)F       ,   N(x,f) 
"^12         Hj^+n^  '  '  ^   '  n  +n-p2(n2) 
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P2^^2> 


=  F   (x,f,)F   (x,f  ) 
n,     i   n»    2 

which  completes  the  proof  of  (4). 

Equation  (4)  has  been  proved  true  which  implies  that 

R*    ^  R     (f)  =  R   (fi)R   (fo)  which  completes  the  proof, 
n^+n^   n^+n^       n^   1  n^   2 

Next  note  that  once  an  algorithm  has  been  found  such  that  R(f)  >  1  and 

p  (s)  <  s  for  s  >  1,  that  ^  R"    ^  ^  R   (f^  )  +  ^  R   (f_)  .   If  it  is  as- 

n,+n_     n^   i       n~   z 

sumed  that  all  optimal  algorithms  have  this  property  from  some  n  on,  then 
0n  R  *  will  be  a  monotonically  increasing  convex  function.  From  the  Hell- 
man-Cover  bound. 


R  *  ^ 
n 

^  R  * 
n 

^afe(£) 

M  Rn* 
n 

-.^(£) 

8/n  Rn* 
If  ^  R  *  is  monotonically  increasing,  then  the  Limit  '^—   exists,  is 

positive,  and  is  bounded  above  by  ^  ■^. 

q 

To  improve  of  the  lower  asymptotic  bound  for  R  *,  consider  the  algorithm 
f  =  <  p  (1) ,  . . .  ,p  (n)>  ea  ,  with  p  (s)  =  1  for  s  =  1, .  .  .  ,k  and  p  (s)  =  s-k  for 
s  =  k+l,...,n;  where  1  ^  k  ^  n.   From  the  recurrance  relationship  for  F, 
we  have 

Vl^''^  =  V2^^^  "  (l-x)x''"^"^Fj^_^(x) 
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F^^^(x)  =  Fj^(x)  -  (l-x)xF^_j^(x) 

By  substitution, 

F^(x)  =  F^(x)  -  (l-x)(x+x  +  .  .  .  +x^'    )  \_i(x)  (5) 

To  establish  an  expression  for  F,  (x) ,  we  will  prove  by  induction  that 

F.(x)  =  [x^'^^-(l-x)^'*"^  ](2x-l)"-^   for  j  =  l,...,k. 

Proo  f : 

2      2 

Note  that  F,  (x)  =  ^'^}'''^      =  ~^  =   1. 
1       2x-l       2x-l 

F  (^)   _  X  -(1-x)   _  2x  -3x  +3X-1    ;l-x+x^ 
^2^""^  "  2x-l     "     2x-l       ^  ''^''   ' 

By  the  induction  hypothesis, 

F.(x)  =  [x^'^^-Cl-x)^'*'^]  (2x-l)"^   for  j  <  k 

Fj_^(x)  =[xJ  -  (l-x)J]  (2x-l)"^ 

From  the  recurrance  relationship, 

F^_j^^(x)  =  F^(x)  -  x(l-x)F^_^(x) 

_  x^"^^-(l-x)^"^^-x(l-x)[x^-(l-x)^l 
2x-l 

KJ+^-(l-x)J+^-xJ+^W+^x(l-x)J+^ 
2x-l 

j+2  ..  \j+2 
x-*   -(l-x)-' 

2x-l 
which  completes  the  proof  of  (6)o 

From  (5)  and  (6) 

k+1  k+1  ,1x^2,       ,  n-k.  .  k  k. 
F  /q  f)  ^  .3 -P    -(l-q)(q+q  +  .  .  .  +q    )  (q  -p  ) 

n   '  q-p 

k+1  k+1  ,  n-k+1   .  -  k  k, 
^  q    -P    +(q      -q)(q  -p  ) 

q-p 

k,   .     n-k+1,  k  k. 
=  P  (q-p)  +  q      (q  -p  ) 
q-p 

k  ,   n-k+1,  k  k.  ,     .-1 
=  p  +  q    (p  -q  )(p  -  q) 
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Similarly, 

^  /   i-\    k    n-k+1  .  k    k.  ,     ,-1     , 
F  (p,f)  =  q  +  P     (p   -  q  )(p  -  q)    and 
n 

k    n-k+1,  k    k.  ,     . -1 
R  /fx  =  P   +  q      (P   -  q  )(p  -  q) 

n^  '^    k    n-k+1,  k    k.  ,     .-1 

q  +  p    (p  -  q  )(p  -  q) 

If  a  =  —  remains  constant,  then 
n 

an         n-cvn+l ,  an         o^n.  ,  .-1 

R  (f)  =  p     +  q (p     -  q    )(p  -  q) 

n  an.         n-an+l,  an         an.  .  .-1 

q      +  p  (p      -  q    )(p  -  q) 


an  -an  nCl-a)  ,  an         an^ 


£^         1   +  p         q    ^         ^  (p         -    q ) 

"l  T     ,      -an  n(l-a),  an         an, 

1  +  q        P  (p        -   q      ) 


p  -  q 
q     p  (p      -  q    ) 


an   ,  n,   -an  -an. 


p    -    q 

q 


(2\         1  +   q"(q   ^"    -  p  ^")    p    -    q 

\q/ 
^  -  n,   -an  -an^         p 

1  +   p    (q  -  p  )    —^ 


n 


an         an 

-E -  q 


/   van   1  +  q  an  an         p   -   q 

=  (s)  E_S1 1 1_ 

\q/  an         an 

1  +  p"    2 — ::-3 E_ 

an  an  p   -   q 

P    q 

,      1-a  V n 

(—3 )      ,  an         an.      — ^ — 

fp\      ^  +^    J^    ^     (P      -q)p-q 


q 


-^  "n  =  {-^)^r  -  r^  -  ,"^^->  -  (-V)" 


But        \      <    I    for   all  a,    so   Limit   U     =  0. 
a  n 

p  n-»<» 

Let   V     =  (-^-^^      ^-'"'  -   -"""^ 

q 

n 


q 
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^     -♦  0  for  all  Q',  but  — ^ —  may  be  greater  than  or  equal  to  1  depend- 


ing on  the  choice  of  a. 


«  ^  =  1. 


In 


a 


=  0;   In  p  -  Ck-  In  q  =  0;  Q-  = 


In  p 
In  q 


If 


Oi 


>   1, 


Inp  -a  lnq>0;  oi  >     "  P 
'^  -1    »     In  q 


If 


a 


<  1, 


lnp-Q'lnq<0;a< 


In  p 
In  q 


0 

Thus  Limit  V  =-^1^    ifo^ 
vi-*°         n 

oo 


< 


i;j 


In  p 
In  q 


Substituting, 

-an 

Limit  R  (-^ 
n\q 
nr»oo     ^ 


Limit 
n-»oo 


1  +  U  — 3— 
n  p  -  q 


I  +  V 


n  p  -  q 


=   0    if  a  > 


In  p 
In  q 


1    .  ^   ^  In  p 

=  1   if  a  < ^ 

In  q 


1  + 


p  -  q 


p  -  q 

2p  -  q 


if  a  = 


In  p 
In  q 


Then  since  R  *  ^  R  (f),  R  «  increases  asymptotically  at  least  as  fast  as 
n      n      n  ■'    '^  ■' 

1"  P 


©  '"  ^  • 
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With  the  previous  assumptions  regarding  the  optimal  form  of  the  algorithm, 

we  can  now  state  that 

In  R  * 

T  •  •*.     n  ^   In  p  1   P   , 

Limit  ^  -; ^  In  ■'^ 

n         In  q    q 
n-»oo  ^    ^ 
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VI.   DETERMINATION  OF  R  *   AND  f>''  FOR  SMALL  n 
j^ 


To  establish  asymptotic  bounds  on  R  *,  it  was  necessary  to  assume  that 
from  some  n  on,  optimal  algorithms  were  such  that  R  *  >  1  and  p(s)  <  s  for 
s  >  1. 

To  test  this  assumption,  R  (f)  was  calculated  for  n  =  1,2,3,  and  4. 
For  each  case  R  *   was  determined  algebraically.   For  the  values  of 
pe  (%,1)  it  was  found  that 

R*  =  R*   =  1        for  any  f 

3 

V  "  ^  "'^  ^3         with  f--^  =  <1,1,2> 
q  +  P 

R^-,  ^  1  -  Pq  -  2pq^  ^i,h  f^-  =  <1,1,2,2> 
1  -  pq  -  2p  q 

The  calculations  used  to  determine  these  values  of  R*   are  at  Appendix  A. 
For  values  of  n  from  5  to  9,  several  values  of  p  >  %  were  chosen  and 
the  search  was  performed  using  an  IBM  360/67  in  double  precision.   The 
program  is  at  appendix  B.   For  the  vicinity  of  1,  values  of  p  of  .99  and 
.999  were  used.   In  the  vicinity  of  %,  the  optimal  algorithm  was  deter- 
mined by  a  Taylor  series  expansion  around  %  +  e,  and  neglecting  terms  of 

2  2 

order  e  .   By  neglecting  terms  of  order  e  , 

F  (%)  -  eF'(%) 

\^''^        F  (%)  +  eF'(%) 
n        n 

To  maximize  R  (f),  minimize    ,i  ,  . 
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Since  F  is  a  polynomial  in  x  of  degree  less  than  or  equal  to  n, 

n 

F  (x)  =  ;  a(n,j)x-^, 
n     z_i 

j=0 

where  a(»,*)  denotes  the  coefficients  of  the  polynomial.   Differentiating, 

n 
f;(x)  =^  ja(n,j)x^"\ 
j=l 


forming   the  ratio  yields. 


n  .    1 


F;(%)  X^   ja(n,j)(%) 


_     2 


^l  ja(n,j)2 


n 


jSo     ^('^'J)^ 


The  program  for  the  search  of  the  ratio  of  these  polynomials  is  at 
Appendix  C. 

The  results  are  summarized  in  Table  I  and  figures  7,8,  and  9.   They 
seem  to  confirm  the  assumptions  made  in  proposition  2  and  further  indicate 
that  the  optimal  form  of  the  algorithm  is  probably  of  the  form 

I       \l,i,,««»,l,Z,...,Z,J,a««,J,M',««««/ 

with  the  lengths  of  the  blocks  of  constant  p (s)  depending  on  n  and  p. 
Computer  run  time  precludes  a  complete  search  of  algorithms  for  n 
much  larger  than  9;  however,  it  may  be  useful  to  assume  that  f*  is  of  the 
form  (l,.. ,1,2,.., ,2,3,...)  and  to  maximize  R  by  manipulating  the  lengths 
of  blocks  of  the  p's.   Such  an  algorithm  is  also  intuitively  appealing. 
Near  the  initial  state  the  information  content  of  an  event  is  low  as 
characterized  by  p (s)  =  1.   As  the  automaton  approaches  the  decision 
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point  (absorption),  p (s)  increases  as  if  the  information  content  of  a 
negative  event  had  increased.   In  some  respects  this  seems  to  be  similar 
to  the  human  decision  process. 
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Figure  7 . 

Natural  logarithm  of  R*  versus  n 
"  n 
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Figure  8 . 
Natural  logarithm  of  R*  versus  n. 
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APPENDIX  A 
Calculation  of  R*  for  n=3  and  n=4. 


For  the  case  when  n  =  3,  f^*  =  (1,1,2),  and 


2    3 

T,  *  _  1  -  2p  +  3p :^_£_ 

^3  ~    ,        3 

1  -  P  +  p 


Proof: 

If  f^^^  =  <1,1,1>,  then  R^-*-^  =  1. 

If  ff )  =  (1,1,2),  then  Rf )  =   ^   '    ^P  ^   ^p^  p^ 

1  -  P  +  P 

3p  >  1  +  2p^     for  all  %  <  p  <  1. 

2        3 

3p^  >  p  +  2p 

2    3  3 

1  -  2p  +  3p  -  p  >  1  -  p  +  p 

1  -  2p  +  3p 2-2.    >  1 

1  -  P  +  P^ 

Therefore  R^^^  >  R^^\  and  R^*  =  ^^      . 

For  the  case  of  n  =  4,  f  *  =  (1,1,2,2),  and 

2  2     3 

R  *  =  1  -  pq  -  2pq      ^   1  -  3p  +  5p   -  2p 

4  2  2     3 

l-pq-2pq        l-p-p+2p 

Proof; 

If  f^^^  =  (1,1,1,1),  then  R^^^  =  1. 

6p  -  4p^  >  2    for  all  %  <  p  <  1 

2     3  2     3 

6p  -  4p  >  2p;   adding  1  -  3p  -  p  +  3p 

2     3  2     3 

1  -  3p  +  5p  -  2p  >  1  -  p  -  p  +  2p 
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2     3 

1  -  3p  +  5p  -  2p 

2     3 
1  -  p  -  p  +  2p 

Thus  R,*  >  R/^^ 
4     4 

If  f^2)  .  <i, 1,1,2),  then  Rp >  =  1  -  3p  +  4p^  -  p^ 

1  -  2p  +  p  +  p 

3         2     4 
6p  +  15p  >  1  +  14p  +  6p       for  all  %  <  p  <  1 

2      4         3     5 
6p  +  15p  >  p  +  14p  +  6p 

adding  1  -  5p  +  6p  -  14p  -  9p  +  9p  -  2p 

1   -   5p  +  12p^  -    14p^  +  6p^  +  3p^   -   2p^ 

>  1   -   4p  +  6p^   -   9p^  +  9p^   -   2p^ 
(1   -   3p  +  5p^   -   2p^(l   -   2p  +  p^  +  p^) 

>  (1  -   3p  +  4p^   -   p^)(l   -   p  -   p^  +  2p^) 

1  -   3p  +  5p^   -   2p^  1   -   3p  +  4p^   -   p^ 

2  3  2  3 

l-p-p+2p  l-2p+p+p 

Thus  R,*  >  R,^^^ 
4  4 

If    £<3)   =   <1,1.1,3),    Rf  >   ,   1   -   3P  ^   5p^   -   3A  P^ 


6p+25p  +14p  >  l+16p  +24p  +4p    for  all  %  <  p  <  1. 
6p  +25p  +14p  >  p+16p  +24p  +4p 
Adding  l-5p+7p^-19p'^-7p^-12p^-7p^-2p^ 

0*3/        Q^T  0*3/        ^^7 

l-5p+13p  -19p  +18p  -12p  +7p  -2p  >  l-4p+7p  -3p  -7p  +12p  -7p  +2p 

(l-3p+5p^-2p^)  (l-2p+2p^-p^+p^)   >   (l-3p+5p^-3pVp^)  (l-p-pV2p^) 

l-3p+5p  -2p    l-3p+5p  -3p  +p 
1-p-p  +2p      l-2p+2p  -p  +p 

Thus  R,  *  >  R,   . 
4     4 
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2        3     4 

Tf   f(^)   -   /I    1   2   1   >      R^^^   -   l-3p+5p   -3p  +p 
ij.    i  —   \i.,j.,^,i.,/,    i\,        —  2      3      4 

l-2p+2p   -p  +p 


C4') 
Thus,    R,*  >  R;   "^ 
4  4 


If   f<^^=   <1, 1,2,3),   Rp >   =   l-3p+6p^-4pV 

^  1-P+P 

10p+14p^  >  2+18p^+4p^ 

10p^+14p^  >  2p\l8p^+4p'' 

Adding  l-4p+8p^-7p'^-7p^-3p^-9p^-2p^ 

l-4p+8p^-7p\3p^-3p^+5p^-2p^  >  l-4p+8p^-5p"^-7pVl5p^-9p^+2p'' 
(l-3p+5p^-2p^(l-p+p'^)  >  (l-3p+6p^-4p\p^)(l-p-p^+2p^) 

l-2p+5p  -2p   >  l-3p+6p  -4p  +p 

2   3  4 

1-p-p  +2p  1"P+P 

Thus  R,*  >  r)    which  completes  the  proof. 
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APPENDIX  B 

//AND12514    JOB    (  2514,  12  42FT  ,  RL22  )  ,  •  AN  DERSON  '  ,  T  I  f^,E  =(  4,  0) 
//    EXEC    FORTCLG, REGION. G0=100K 
VPnRT  .SYSIN    DD    '■^■• 

;HPLICIT  REAL*8(A,BtC,D,R,S) 

DIMENSION  A(ll)  ,B(11) 

A(l)=l. 


30 


B(l)=l. 

A{2)=I. 

B(2)=l. 

READ(5,2000,END=20)C 

WRITE(6,3005)    C 

D=l.-C 

A(3)=l.-C*'0 

B(3}  =  l.-C'5.-D 

SMAX=0. 

DO    10    I 3=lt2 

DO     10    I  4=  It  3 

DO    10     15=1,4 

DO    10    16=1,5 

DO    10    17=1,6 

DO    10     18=1,7 

DO    10    19=1,8 

A(4)=A(3)-C**I3^D^A(3-I3) 
B(4)  =  B(3  )-D'^*I3*^C^B(3-I3) 
A{5)  =  A(  4)-C^^I4*D'J  A(4-I4) 
B  (  5  )  =  B  (  4 )  -  D  - 'i^  1 4  ^  C  •- B  (  4  -  I  4 ) 
A(6)=A(5)-C^•^■I5^D-^A(5'I5) 
B(6)  =  B(  5)-D^/*I5^C4^B{5-I  5) 
A  ( 7  )  =  A  {  6  )  -C-;  *  16 *■  D*  A  (  6-  I  6  ) 
B(7)  =  B(  6)-D'/?I6^C=^B(6-I6) 
A(8)  =  A(  7)-C-^^I7^D*A(7-I7) 
B(8)=B(7}-D^*I7t:C^B(7-I7) 
A(9)=A(  8)-C^i^I8(D*A(8-I8) 
B{9)=B(  Bl-D'f^vis^  CvBCa-IS) 
A(10)=A  (9  )-C>*I9-^^D--^-A{9-I9) 
B  (  1 0  )  =B  (  9  )  -D  ?>.  I  9  ^  C^^-  B  (  9-1  9 ) 
R=B(10)/A( 10) 
IF(R. LT.SMAX)     GO    TO    10 
SMAX=R 
K3=I3 
K4  =  I4 
K  5=  I  5 
K6=I6 
K7  =  I7 
K8=I8 
K9=I9 
10    CONTINUE 

WRITE    (6,3000)     S^1AX 

WRITE(6,3001 )  K3,K4,K5,K6,K7,K8,K9 

3001  FORMAT( T10,7I6,//) 
GO  TO  30 

2000  FORMAT( F6 .4) 

3000  FORMAT  (T10,F15. 7) 

3005  FORMATiTlO, 'F=  ',F6.4) 
20  STOP 
END 
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APPENDIX  C 

IMPLICIT    INTEGER*4(RtS) 

DIMENSIOVi    R(  lit  11)  t  S(  11,11) 

DATA    R/121^0/,S/121-^0/ 

R(2»2)=l 

R(3,2)=l 

S(2,2)=l 

S(3,2)=l 

S(2,3)=-l 

S(3t3)=-1 

A2=99999. 

12=1 

DO   20    K=l ,12 

20  R(4,K)  =  R(  3,K) 
M=I2+1 

DO   21    K=M,11 

21  R(4,K)  =  R(3,K)-S(  3-I2,K-I2) 
DO    22    K=2,ll 

22  S(4,K)=R(4,K)-R(4,K-1 ) 
DO    400    13=1,2 

DO    30    K=l ,  13 

30  R{5,K)=R(4,K) 
M=I3+1 

DO    31    K=M, 11 

31  R(5,K)=R(4,K)-S(4-I3,K-I3) 
DO    32    K=2,ll 

32  S(5,K)=R{5,K)-R(5,K-1) 
DO    400     14=1,3 

DO    40    K=1,I4 

40  R(6,K)=R(5,K) 
M  =  I4^-l 

DO    41    <=M,11 

41  R(6,K)=R(5,K)-S(5-I4,K-I4) 
DO    42    K=2,ll 

42  S{6,K)=R(6,K)-R(6,K-1) 
DO   400     15=1,4 

50  R(7,<)=R(6,K) 
M=I5+1 

00    51    K=M,11 

51  R(7,K)=R(6,K)-S( 6-I5,K-I5) 
DO    52    K=2,ll 

52  S(7,K)=R{ 7,K)-R(7,K-1 ) 
DO    400     16=1,5 

DO   60    K=l , 16 

60  R(8,K)=R( 7,K) 
M=I6+1 

DO    61     K=M,11 

61  R(8,K)=R( 7,K)-S( 7-I6,K-I6) 
DO    62    K=2, 11 

62  S(8,K) =R(8,K)-R(8,K-1 } 
DO    400    17=1,6 

DO    70    K=l, 17 

70  R(9,K)=R(8,K) 
M=I7+1 

DO    71    K=M,11 

71  R(9,K)=R(8,K)-S(8-I7,K-I7) 
DO    72    K=2, 11 

72  S(9,K)=R(9,K)-R(9,K-1 ) 
DO    400    18=1,7 

DO    80    K=l, 18 

80  R(10,K) =R(9,K) 
M=I8+1 

DO    81    K=M, 11 

81  R(10,K) =R(9,K)-S(9-I3,K-I8) 
DO    82    K=2, 11 

82  S (10,K) =R( 10,K)-R(10,K-1) 
DO    400    19=1,8 

DO    90    K  =  l,  19 

90  R(11,K)=R(10,K) 
M  =  I9+1 

DC    91    K=M,  11 

91  R(11,K) =R(10,K)-S(10-I9,K-I9} 
DO    92    K=2,ll 
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92    S(11»K)=R(  11,<)-R(  11»K-1) 
R1=0 
R2=0 
N7  =  9 

00   27    J=1tN7 
J1=J 
J2=N7-J1+1 

27  R1  =  R1  +  J1<-21'*J2*R(11  ,Jl+2} 
N7=N7+1 

DO    28    J=1,N7 
J1  =  J 
J2=N7-J1 

28  R2  =  R2+2-t*J2tR(ll,  Jl  +  1) 
A1  =  R1 

A1=A1/R2 

IF(A1 .GT.A2)  GO  TO  400 
A2=A1 
L2=I2 
L3  =  I3 
L4=I4 
L5=I5 
L6  =  I6 
L7=I7 
L8=I8 
L9=I9 

R12=R( 11,2) 
R13  =  R{ lit  3) 
R14=R(11t4) 
R15=R( 11,5) 
R16=R(11,6) 
R17=R(11,7) 
R18=R(  11,  8) 
R19=R(il,9) 
R110=R( 11 ,10) 
R111=R(  11,  11) 
CONTINUE 
WRITE( 6,2001) 
1,R111 
WRITE (6 ,2002) 
FORMAT ( TIO, 'COEFFICIENTS' 
F0RMAT(T10,' INDEX    VALUES' 
STOP 
END 


400 


2001 
2002 


R12,R13,R14,R15,R16,R17,R18,R19,R110 


L2,L3  ,L4,L5 


,L6,L7,L8,L9, A2 
,1016) 


,8I5,/,T10,»MIN=»,F14.8,//) 
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